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Appropriate and Useful: Principal Attitudes Toward K—12 Standards and Policies 


Adam K. Edgerton, Jill McLaughlin & Laura M. Desimone 


Using state-representative surveys, we investigated how familiar Texas, Ohio, Kentucky, and Massachusetts principals are with their state’s K-12 standards, as well as the strength of their policy environments, using a policy attribute theory. 
We found relatively high levels of buy-in toward the idea of standards but only a general familiarity with the standards themselves. Principals are least familiar with the standards in mathematics, and high school principals are significantly 
less familiar with their state’s standards than elementary principals. Using ordinary least squares (OLS) regression, we found no relationship between student proficiency and principal familiarity, but we did find a significant and positive 
relationship between stronger policy environments and greater familiarity among principals in Texas. 


Among the many challenges facing principals today is the need to be instructional leaders across content areas, even if they lack experience teaching certain subjects (Hitt & Tucker, 2016; Quebec Fuentes et al., 2015). For instance, principals 

who formerly taught math still must support their English language arts (ELA) teachers, regardless of the principal’s level of familiarity with ELA content. Principals also must evaluate their instructional staff based on their coverage of content 

standards—standards that are mandated by state education agencies (SEAs). For decades, standards-based reform has operated under the theory that if all teachers teach a common set of standards, student achievement will rise (Dee & Jacob, 
2011; Desimone et al., 2019; Smith & O’Day, 2011; Spillane, 2009). Though the evidence for using standards to raise student performance is mixed (see Polikoff, 2017; Song et al., 2019), the institutional logic of using standards as a mechanism 

for ensuring instructional quality persists across the United States (Edgerton, 2020). 


Standards-based policies could plausibly push principals to build familiarity with the content taught across subjects and grade levels, particularly in commonly tested subjects such as math and ELA. Principals who are more familiar with these 
standards could become more competent instructional leaders. They have incentives to do so, given that they are often held accountable for their school’s test results, though not to the same extent as individual teachers (Edgerton & Desimone, 
2019). Prior research supports the notion that principals’ instructional leadership and their framing of learning standards are crucial to successful implementation of standards-based policies (Eilers & D'Amico, 2012; Stosich, 2017). Effective 
instructional guidance from principals is also more critical to implementation than the initiative of individual teachers (Stosich, 2016). 


However, both the demographic context and the policy environments in which principals operate influence their ability to perform their instructional leadership duties (Grissom et al., 2013). Prior evidence shows that principal quality, including 
their level of experience and education, is unevenly distributed among high- and low-poverty schools, with high-poverty schools more frequently led by less experienced leaders (Clotfelter et al., 2007; Loeb et al., 2010). Given these factors, 
this research note examines how policy environments and demographic context relate to principal knowledge of standards within the context of student learning (as measured by proficiency on state standardized tests). We shed light on how 
well-crafted standards-based policy might support principals in their effort to facilitate the teaching of K-12 standards. 


We analyzed principal survey responses from four states—Texas, Ohio, Kentucky, and Massachusetts—using a policy attributes theory to quantify their policy environments (Porter, 1994; Porter et al., 1988). We examined whether principals’ policy 
environments predict their familiarity with K-12 standards and whether this familiarity is related to the percentage of students in their school who are proficient or the percentage of students who qualify for free and reduced lunch (FRL). 
Throughout this research note, we use “standards” to refer to the latest K-12 standards in every state as of 2016, given that states are continually drafting, revising, and implementing their own versions of what were previously the Common 
Core State Standards (CCSS; Peterson et al., 2016). In Texas, we mean the Texas Essential Knowledge and Skills (TEKS); in Ohio, the Ohio Learning Standards; in Kentucky, the Kentucky Academic Standards; and in Massachusetts, the 
Massachusetts Learning Standards. 


We answer the following research questions: (1) To what extent do principals experience specific, consistent, authoritative, powerful, and stable policies related to their state’s K-12 state standards? (2) How familiar are principals with their 
state’s K-12 standards? (3) Do principals’ policy environments and the percentage of students on FRL predict familiarity with their state’s K-12 standards? (4) Are principals’ policy environments and their familiarity with their state’s K-12 
standards related to student proficiency? 


SURVEY DESIGN AND MEASURES 


We surveyed four states in 2016—Texas, Ohio, Kentucky, and Massachusetts—as part of a multifaceted study of standards-based reform. We developed the survey through a process of expert review and piloting, and we borrowed items from 
nationally validated surveys when possible (Desimone & LeFloch, 2004). American Institutes for Research contacted sample schools within the Ohio and Texas districts that agreed to participate. The surveys in Kentucky and Massachusetts 
occurred in conjunction with their SEAs. As a result, some of the policy attribute items were cut from the survey. Consequently, only with Texas and Ohio data were we able to (1) regress student proficiency and familiarity on the policy 
environment and (2) regress familiarity on the policy environment. We provide descriptive findings, however, across all four states. 


Eligible districts in Texas, Ohio, and Kentucky were identified via a stratified sampling design to ensure that findings would represent states as a whole. We paid special attention to including districts with substantial numbers of English learners 
and students with disabilities, as well as each region of the state and each geographical category (rural, suburban, and urban). In Ohio and Texas, respectively, 155 and 121 eligible districts were contacted. Among the eligible districts, 49 and 
53, respectively, agreed to participate, and 42 districts completed the survey. In Kentucky, 89 districts were included and surveyed. In Massachusetts, all 289 school districts were surveyed in partnership with the SEA’s existing annual survey 
(there was no stratified sampling design). Of the 185 eligible elementary and high school principals in Ohio, 111 responded (conditional response rate: 60%), 149 of 211 eligible principals in Texas responded (71%), 179 of 354 eligible principals in 
Kentucky responded (51%), and 963 of 1,851 principals in Massachusetts responded (52%). To correct for nonresponse bias, we used survey jackknife procedures in Stata with demographic weights when presenting our results. Our results are 
representative of states as a whole, not individual districts. 


In Texas, Ohio, and Massachusetts, we asked elementary and secondary principals about their familiarity with their state’s math and ELA standards, to which they replied: 1 (“Il am unfamiliar or only slightly familiar”); 2 (“lI have a general 
understanding, but | am not familiar enough with all of the details”); or 3 (“I have a detailed understanding”). We did not sample middle school principals because of resource constraints and because we expected a sharper contrast between 
elementary and secondary principals. 


We measured the strength of policy environments through five policy attributes: specificity, or how detailed standards-based policies are; authority, or how much buy-in policies receive; power, or the use of rewards and sanctions to reinforce 
policies; stability, or how long policies last; and consistency, or how policies align with other policies and beliefs. According to the policy attribute theory, the stronger the policy environment, as measured by these five attributes, the more 
likely a policy is to be implemented with fidelity and to succeed. For this reason, we used summative scales as a measure of strength. The theory posits that when implementing K-12 standards, districts will have more success if their policies 
and programs are specific, are consistent with already existing policies, build buy-in among implementers, have appropriate rewards and sanctions, and are stable over time. 


To measure specificity, we asked principals to what extent their teachers had received specific guidance from the district about the order in which to teach standards and how much time to spend on each standard. To measure consistency, we 
asked principals a series of questions about whether they had textbooks, curricular materials, assessments, and other materials aligned to their state’s standards available to them. To measure authority, we asked principals whether they 
believed their state’s standards were appropriately rigorous and flexible, offered a manageable number of topics to teach each year, and included all important concepts. To measure power, we asked principals a set of questions about whether 
they were publicly rewarded when their students performed well on state tests or punished when they did not. To measure stability, we asked principals how long they anticipated that their state’s K-12 standards and other related policies would 
stay in effect. Responses were on a Likert scale from 1 to 4, with 4 representing the strongest possible response, and 1 representing the weakest. The five policy attribute constructs—specificity, consistency, authority, power, and stability—are 
multi-item composites with an acceptable internal reliability (Cronbach’s a of 0.80 or above; Edgerton & Desimone, 2019). A full list of these items can be found in Appendix. 


METHODS 


For achievement outcomes, we used publicly available measures of math and ELA achievement (percent of students achieving proficiency). In Ohio, this was Ohio State Test, and in Texas, this was State of Texas Assessments of Academic 
Readiness (STAAR). We matched principals to average student achievement (% proficient) using their unique NCES identification number for their schools. To preserve respondents’ anonymity, we do not identify these schools. 


To obtain the estimates used in this research note, we used OLS regression after merging publicly available data from the Texas and Ohio SEAs. The equation for Research Question 3 is as follows: 


= a+ BAtt + BExp + BElem + BFRL + BPro 


where y = familiarity with their state’s math or ELA standards, Att is the cumulative strength of the policy environment (ranked 1-20), Exp is years of experience as a principal, Elem is a binary indicator of whether a principal is at the 
elementary or high school level, FRL is the percentage of students on free and reduced lunch, and Prof is the percentage of students proficient in their school. 


For Research Question 4, we used a slightly modified equation, where proficiency (Prof) is replaced with either math or ELA familiarity (Fam): 


y= a+ BAtt + BExp + BElem + BFRL + BFam 


References to “significance” in the text that follows indicate p < 0.05. 
FINDINGS 
Research Question 1: To what extent do principals experience specific, consistent, authoritative, powerful, and stable policies related to their state’s K-12 state standards? 


In Figure 1, we present the averages of the attributes for principals in each state, as well as the extent of their familiarity with their state’s standards. For specificity, with averages between 2.60 and 2.94, principals in all four states felt that 
districts were providing somewhat specific guidance on the order in which standards should be taught and how much time should be spent on each topic. For consistency, with scores between 2.52 and 2.92, principals in all four states felt that 
assessments, curricula, and textbooks were aligned (though not strongly aligned) to the standards. Authority, on average, was relatively high, with scores between 2.95 and 3.36. Authority represents the extent to which principals believed the 
standards were appropriate, flexible, and rigorous enough for all students, including students with disabilities (SWDs) and English learners (ELs). These relatively high authority scores mean that principals, on average, somewhat agreed (3 out of 
a possible 4) that state tests provided useful information. In contrast, for power, averages were far lower, falling between 1.96 and 2.55; principals somewhat disagreed that they were punished or rewarded for student scores on these tests. For 
stability, cross-state averages between 2.58 and 2.83 demonstrate that principals expected their state standards and annual assessments to last for another four years. 
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Figure 1. Principal policy attributes and reported familiarity with the standards 
Looking across the four states surveyed, there was moderate specificity, consistency, and stability. Scores were strikingly similar across the states, and most differences were not statistically significant. Two distinctions were both meaningful in 
terms of the magnitude of the difference and statistical significance at the 0.05 level. First, Massachusetts principals reported significantly higher authority (3.36) than the other three states. Second, Kentucky principals reported significantly 
lower power than the other three states (1.96). But across all states, relatively high authority numbers revealed that principals were buying into their state’s standards. Principals reported relatively low power, indicating that they were 
experiencing relatively few rewards and sanctions. 
Research Question 2: How familiar are principals with their state’s K-12 standards? 
In Massachusetts, 55% of principals reported detailed familiarity with their state’s standards in ELA (represented by 3 on a 1-3 scale), and 59% reported detailed familiarity with their state’s math standards. In Texas, 48% of principals reported 
detailed familiarity with their ELA standards, and 43% reported detailed familiarity with their math standards. In Ohio, 48% of principals also reported detailed familiarity with their ELA standards, but this number dropped to only 30% of 
principals reporting detailed familiarity with their math standards. This 30% is statistically significantly different compared with Massachusetts and Texas principals’ ELA and math familiarity and compared with Ohio principals’ ELA familiarity. 


In Figure 1, the overall averages are lowest in secondary math, with scores closer to 2 in both Texas (2.20) and Ohio (2.16), which are both significantly lower than the Massachusetts average (2.52). Furthermore, in both math and ELA in all 
three states, high school principals reported significantly less familiarity than elementary principals. 


Research Question 3: Do principals’ policy environments and the percentage of students on FRL predict principals’ familiarity with their state’s K-12 standards? 

In Table 1, the percentage of FRL students in a school does not significantly predict principal familiarity with standards (as shown in columns 3, 4, 7, and 8). However, a stronger policy environment does predict significantly greater principal 
familiarity with standards in Texas in both ELA and math (+0.054 and +0.052, respectively, in columns 3 and 4). Though the magnitude of this relationship seems small, a Texas principal in the strongest policy environment (20 out of a possible 20) 
moves a full category upward in their knowledge of the standards, as compared with a Texas principal working in the weakest possible environment (1 out of a possible 20), after controlling for the economic background of students in the school 
and the principal’s years of experience. 


Table 1. OLS Regression Results for Texas and Ohio Achievement and Principal Familiarity With the Standards 


(1) (2) (3) (4) (5) (6) (7) (8) 


TX ELA TX Math TX ELA Fam. TX Math Fam. OH ELA OH Math OH ELAFam. OH Math Fam. 
PAtt. (1-20) -0.031 -0.199 0.054" 0.052" 0.601 1.234 0.013 0.034 
(0.627) (0.643) (0.019) (0.021) (0.771) (0.904) (0.039) (0.028) 
Yrs of Exp. 0.056 -0.035 0.006 0.001 0.038 0.041 -0.005 -0.008 
(0.344) (0.284) (0.008) (0.011) (0.391) (0.301) (0.019) (0.008) 
Elementary 2.300 10.233" 0.286" 0.307" 1.534 20.993" 0.249 0.342" 
(3.069) (3.447) (0.099) (0.079) (3.269) (4.359) (0.128) (0.124) 
E Fam (1-3) 3.303 -1.450 
(2.687) (3.390) 
M Fam (1-3) 2.313 -7.245 
(2.547) (4.388) 
% FRL -0.489". -0.377 -0.000 -0.002 -0.405 -0.414° 0.002 -0.003 
(0.042) (0.056) (0.003) (0.002) (0.085) (0.082) (0.004) (0.003) 
ELA % Prof 0.004 -0.002 
(0.004) (0.006) 
Math % Prof 0.002 -0.007 
(0.003) (0.004) 
Constant 72.514 62.884. -0.694 -0.503 72.056. 49.923. 0.280 0.184 
(7.584) (8.004) (0.406) (0.364) (11.056) (12.966) (0.581) (0.458) 
N 148 148 148 148 100 100 100 100 
p2 0.633 0.476 0.179 0.177 0.563 0.607 0.094 0.124 


Note. Standard errors in parentheses. “PAtt” is a summative score of the policy environment (maximum score of 20); “Elementary” is a binary indicator of whether a respondent was an elementary school principal (as opposed to secondary); “E 
Fam” is the level of familiarity with their state’s ELA standards; and “M Fam” is the level of familiarity with their state’s math standards. FRL = free and reduced lunch; ELA = English language arts. 


D < .05. "Dp < .01. 
Research Question 4: Are principals’ policy environments and their familiarity with their state’s K-12 standards related to student proficiency? 
After controlling for the percentage of students who qualify for FRL in the school, neither familiarity with the standards nor policy environments predicted student proficiency (see columns 1, 2, 5, and 6). We were not surprised by these null 
findings because they were consistent with the existing literature indicating that teachers are the more important in-school determinant of achievement (Rivkin et al., 2005). Additionally, the literature is clear that credibly isolating the impacts 
of principals is quite difficult even with longitudinal data sets (e.g., Branch et al., 2012; Grissom et al., 2015). Although we cannot make causal claims, finding no relationships is further evidence of weak coupling (see Weick, 1976) between 
principal familiarity with standards and teachers’ implementation of them. 

DISCUSSION 
High school principals across Texas, Ohio, and Massachusetts, as compared with surveyed elementary principals, reported having only a general familiarity with their state’s standards. It is worth emphasizing the notable similarities across states 
in principals’ experiences with the standards policy environment. This similar finding of only general familiarity, occurring in 2016 after these states had already spent at least 5 years implementing their latest ELA and math standards, suggests 
that we should continue to invest in principal professional development (PD) on the standards—as well as evaluate the rigor of existing opportunities. The standards-based PD these principals received may have been ineffective or inadequate, 
given that much of the policy focus during this period remained on teacher PD. Teachers, in contrast, are less likely to want additional PD that builds their familiarity with the standards; they believe that they understand the standards well, but 
they want more PD on how to differentiate instruction for SWDs and ELs (Edgerton & Desimone, 2018). 
On a more encouraging note, principals in higher FRL schools are as likely to be familiar with the standards as principals in more affluent schools. This finding stands in contrast to prior research indicating that principals are unevenly distributed 
by quality among high- and low-poverty schools (Grissom et al., 2015). Though we did not measure principal quality in terms of their ability to impact student achievement, we did not find that principals in poorer schools were significantly less 
familiar with their state’s standards. We consider knowledge of the standards to be a necessary first step toward effective instructional leadership. Without this knowledge, we do not see how principals can improve teacher effectiveness 
because they cannot serve as instructional leaders and build knowledge of the instructional core through professional development (Stosich, 2017). On a more negative note, secondary school principals, regardless of the family incomes of their 
student population, know significantly less about the standards when compared with elementary principals. High school leaders may need more training on math and ELA standards and may only have knowledge of the content area in which they 
previously taught. 
The significant predictors in Texas suggest that a principal’s knowledge of standards can be related to specific, authoritative, powerful, consistent, and stable policies. Similarly, higher authority in Texas among teachers significantly predicts 
more coverage of ELA standards (Edgerton & Desimone, 2018). Our findings set the stage for future work that could establish a causal link between increased principal knowledge and increased student success, which is beyond the scope and the 
resources of this study. In Texas, stronger policies may be building instructional cohesion at both the classroom and the school level through increased familiarity with the standards among both teachers and principals. Thus, strong state policies 
may have some capacity to improve principal knowledge of math and ELA standards. Similarly, Massachusetts has a significantly higher authority average than the other three states, and it also frequently tops the national rankings of states with 
strong public education systems (Ziegler, 2018). Texas and Massachusetts offer important insights as we look for high-quality models of K-12 standards implementation. 
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Appendix 
Principal Survey Questions 


This appendix details the principal survey questions that apply to each scale in this report. “CCR standards” refers to the name of the standards in each state—in Texas, the Texas Essential Knowledge and Skills (TEKS); in Ohio, the Ohio Learning 
Standards; in Kentucky, the Kentucky Academic Standards; and in Massachusetts, the Massachusetts Learning Standards. 


Specificity 


The scale for specificity was created using the average of question 18. 
0 - don’t know, 1 - disagree strongly, 2 - disagree somewhat, 3 - agree somewhat, 4 - agree strongly 


Question 18 
Please indicate your level of agreement with the following statements: 
My teachers have received specific guidance from my district on the order in which they should teach content area in {{CCR standards}} for ELA. 
My teachers have received specific guidance from my district on how much time they should spend on each content area in {{CCR standards}} for ELA. 
My district has provided teachers in my school with lesson plans aligned with {{CCR standards}} for ELA. 
My teachers have received specific guidance from my district on the order in which they should teach content area in {{CCR standards}} for Mathematics. 
My teachers have received specific guidance from my district on how much time they should spend on each content area in {{CCR standards}} for Mathematics. 
My district has provided teachers in my school with lesson plans aligned with {{CCR standards}} for Mathematics. 
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Consistency 


The scale for specificity was created using the average of questions 20 and 21. 
0 - Not applicable, 1 - Don’t know, 2 - not at all aligned, 3 - somewhat aligned, 4 - aligned, 5 - strongly aligned 


Question 20 
Please indicate your opinion on the degree to which the following are aligned to CCR standards for ELA. 
a __ The ELAsection of the state test 
b __ District-mandated summative assessments 
c Formative or diagnostic assessments selected or created by your school 
d___ Formative or diagnostic assessments used district-wide 
e _ English/language arts textbooks used in your school 
f | English/language arts curriculum selected or developed by your district 
g Professional development activities that you have participated in this year 
h The feedback | provide to teachers from their classroom observations 
Question 21 
Please indicate your opinion on the degree to which the following are aligned to CCR standards for mathematics. 
a __The math section of the state test 
b __ District-mandated summative assessments 
c Formative or diagnostic assessments selected or created by your school 
d Formative or diagnostic assessments used district-wide 
e Mathematics textbooks used in your school 
f | Mathematics curriculum selected or developed by your district 
g Professional development activities that you have participated in this year 
h The feedback you provide to teachers from their classroom observations 
Authority 


The scale for authority was developed using the average of survey questions 6, 7, 8, 9, 16, and 17. 
0 - don’t know, 1 - disagree strongly, 2 - disagree somewhat, 3 - agree somewhat, 4 - agree strongly 


Question 6 


Please indicate your agreement with the following statements. 
College and career readiness (CCR standards) for ELA set appropriate expectations for student learning at each grade level. 


a 
b CCR standards for ELA make learning relevant to students’ everyday lives. 
c Since {{state name}} started implementing CCR standards for ELA, teachers in my district have made significant instructional shifts to tailor instruction to those standards. 
d__ Results from the ELA portion of the state test provide valuable information about how well students in my school are mastering the state standards. 
e | use results from the ELA portion of the state test to inform my school’s improvement planning. 
f | | use results from the ELA portion of the state test to inform teacher evaluations in my school. 
g | use results from the ELA portion of the state test to inform professional learning decisions in my school. 
h CCR standards for ELA are appropriate for English language learners. 
ij CCR standards for ELA set appropriate expectations for students with disabilities’ learning (including those with mild learning disabilities but excluding those with severe or profound disabilities). 
Question 7 
a CCR standards for Mathematics set appropriate expectations for student learning at each grade level. 
b CCR standards for Mathematics positively affect how well students are prepared to compete in the workforce. 
c CCR standards for Mathematics make learning relevant to students’ everyday lives. 
d__ Since {{state name}} started implementing CCR standards for Mathematics, teachers in my school have made significant instructional shifts to tailor instruction to those standards. 
e _ Results from the mathematics portion of the state test provide valuable information about how well students in my school are mastering CCR standards for Mathematics. 
f | use results from the mathematics portion of the state test to inform my school’s improvement planning. 
g __| use results from the mathematics portion of the state test to inform teacher evaluations in my school. 
h | use results from the mathematics portion of the state test to inform professional learning decisions in my school. 
ij CCR standards for Mathematics are appropriate for English language learners. 
j CCR standards for Mathematics set appropriate expectations for students with disabilities’ learning (including those with mild learning disabilities but excluding those with severe or profound disabilities). 
Question 8 
a CCR standards for ELA exclude important content that students should learn. 
b CCR standards for ELA provide a manageable number of topics to teach in a school year. 
c CCR standards for ELA give educators the flexibility they need to help students who are below grade level. 
d CCR standards for ELA are more rigorous than the previous state standards. 
Question 9 
a CCR standards for Mathematics exclude important content that students should learn. 
b CCR standards for Mathematics provide a manageable number of topics to teach in a school year. 
c CCR standards for Mathematics give educators the flexibility they need to help students who are below grade level. 
d CCR standards for Mathematics are more rigorous than the previous state standards. 
Question 16 
a___| have made teaching to CCR standards for ELA a major priority in my school. 
b My district has made teaching to CCR standards for ELA a major priority. 
c My state has made teaching to CCR standards for ELA a major priority. 
Question 17 
a___| have made teaching to CCR standards for Mathematics a major priority in my school. 
b My district has made teaching to CCR standards for Mathematics a major priority. 
c  Mystate has made teaching to CCR standards for Mathematics a major priority. 
Power 


The scale for power was developed using the average of question 19. 
0 - don’t know, 1 - disagree strongly, 2 - disagree somewhat, 3 - agree somewhat, 4 - agree strongly 


Question 19 


Please indicate your level of agreement with the following statements: 
a___ District leaders publicly reward or recognize principals in this district for exemplary leadership practices aimed at implementing {{CCR standards}}. 
b District leaders publicly reward or recognize principals in this district for exemplary student achievement gains. 
c There are negative repercussions for me if students in my school do not perform well on the state test. 


Stability 


The scale for stability was developed using the average of question 22. 
1 = 1-2 years, 2 = 3 years, 3 = 4 years, 4 = 5+ years 


Question 22 


Including this current school year, how long do you believe each of the following will remain in effect? 
a CCR standards for ELA 
b CCR standards for Math 
c Thestate test 
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